Contemporary Issues In Education Research - Fourth Quarter 2008 


Volume 1, Number 4 


Evaluating The Effectiveness Of A Course- 
Objective Writing Developmental Teacher 

Training Program 

Russell Baker, Jacksonville University, USA 
Gina M. Almerico, University of Tampa, USA 
Barry Thornton, Jacksonville University, USA 


ABSTRACT 

One of the challenges for educators is composing well-written, specifically stated outcomes of 
student learning. Developmental teacher training is a program designed to improve the 
effectiveness of university instructors. The program focuses on improving course methodology by 
guiding instructors through the development of correctly written and structured course objectives, 
effective and efficient delivery methodology including implementation of technology as a delivery 
medium, and proper outcomes evaluation of course effectiveness measured against the course 
objectives. This paper evaluates the effectiveness of the training. The outline used in the training is 
included in the appendix. 


INTRODUCTION 

major component of the training program is developing instructors' course-objective writing skills. 
The American Association of Collegiate Schools of Business (AACSB) accredited business 
schools require course objectives to be written in compliance with Bloom's Taxonomy cognitive 
domain. Bloom's cognitive domain defines the following six distinct levels of learning (Bloom, 1956): 

1. Knowledge: The remembering or recalling of appropriate previously learned information. 

2. Comprehension: Grasping or understanding the meaning of informational materials. 

3. Application: The use of previously learned information in new and concrete situations to solve problems 
that have single or best answers. 

4. Analysis: The breaking down of informational materials into their component parts, examining and trying 
to understand the organizational structure of the information to develop divergent conclusions by 
identifying motives or causes, making inferences, and/or finding evidence to support generalizations. 

5. Synthesis: Creatively or divergently applying prior knowledge and skills to produce a new or original 
whole. 

6. Evaluation: Judging the value of material based on personal values/opinions, resulting in an end product, 
with a given purpose, without real right or wrong answers. 

Bloom stipulated that specific verbs are to be used in course objectives to delineate which level of learning 
the objective addresses. Course objectives must be written to include a statement of the learning objective, a 
statement of the conditions under which the learning will take place, and a measurement of successful 
accomplishment. This criterion is more commonly referred to as the behavior-condition-degree, or BCD, criterion 
(University of Washington). Anyone unfamiliar with Bloom's Taxonomy is referred to Bloom's Taxonomy of the 
Cognitive Domain (1956) or any of the myriad of articles and books published on the subject since the original 
publication. 

To improve the objective-writing skills of university instructors, the Developmental teacher training 
program provides instruction on the basic concepts of Bloom's Taxonomy and verb utilization in objectives. This is 
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intended to help instructors focus their course on specific learning objectives. Thus, under the premise that properly 
written objectives focus course instruction towards accomplishment of the desired learning outcomes. 
Developmental teacher training should help instructors attain higher levels of learning. 

PURPOSE 

The purpose of this research is to determine the effectiveness of Developmental Teacher Training. Course 
objectives taken from course syllabi were evaluated to determine the Bloom's Taxonomy level for each objective. 

METHODOLOGY 

Participants in a recent program were asked to bring course objectives from one of their courses to the 
training. At the onset of training, the instructors were asked to evaluate the taxonomy level of their objectives. The 
instructors were given the above definitions of the taxonomy levels, but no other training or information was 
provided. Subsequent to instructor self evaluation. Bloom’s Taxonomy academic experts established the course 
objectives’ taxonomy levels using the Comprehensive Bloom’s Taxonomy Verb List (Almerico and Baker, 2005). 
The instructors' self assessments were compared with the expert assessment to determine the Self-Assessment 
Accuracy (SAA). The SAA was evaluated at the beginning and at the end of Developmental teacher training in a 
pretest/posttest repeated measure design. 

The SAA is a continuous variable measured from 0 to 1. A perfect Self-Assessment Accuracy is one. SAA 
is calculated using the following procedure: 

1. A point value is assigned to each objective based on the Bloom's Taxonomy learning level it represents. 
Knowledge=l, Comprehension=2, Application=3, Analysis=4, Synthesis=5, Evaluation=6. Point values are 
automatically assigned by the B-CAT software. Point values are assigned to instructor self-evaluated 
objectives based on the learning levels specified by the instructor. 

2. The absolute value of the difference between B-CAT evaluation and instructor evaluation is calculated for 
each objective. 

3. The average difference is calculated for all objectives. This is the average instructor error (AIE). The 
maximum value of average instructor error is five. 

4. SAA is calculated by the following formula: SAA = 1 - AIE/5. Thus, total error (unlikely) on the part of an 
instructor would result in a score of zero. Total correctness on the part of the instructor would result in a 
score of one. 

5. Separate SAA values were calculated for before training and after training for each instructor. 

Thirty university instructors participated in the program. The instructors were from two midsize, private, 
southern universities. All of the instructors had less than five years of university-teaching experience and were first¬ 
time participants in the program. Ten instructors teach in the college of education, ten instructors teach in the college 
of business, and ten instructors teach in the college of liberal arts. SAA values before and after training, categorized 
by college, were calculated using a Microsoft Excel spreadsheet. These values were then read into Minitab for data 
analysis. Table 1 shows the data as read into Minitab, and Table 2 shows the descriptive statistics of interest. 
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Table 1: SAA Scores by College 


SAA Before 

SAA After 

Difference 

College 

0.83 

0.95 

0.12 

Education 

0.89 

1.00 

0.11 

Education 

0.78 

0.96 

0.18 

Education 

0.81 

0.90 

0.09 

Education 

0.74 

0.87 

0.13 

Education 

0.92 

0.94 

0.02 

Education 

0.88 

0.92 

0.04 

Education 

0.85 

0.88 

0.03 

Education 

0.79 

0.91 

0.12 

Education 

0.90 

1.00 

0.10 

Education 

0.74 

0.90 

0.16 

Business 

0.76 

0.93 

0.17 

Business 

0.73 

0.88 

0.15 

Business 

0.62 

0.90 

0.28 

Business 

0.58 

0.92 

0.34 

Business 

0.66 

0.80 

0.14 

Business 

0.78 

1.00 

0.22 

Business 

0.94 

1.00 

0.06 

Business 

0.86 

0.90 

0.04 

Business 

0.90 

0.92 

0.02 

Business 

0.52 

0.88 

0.36 

Liberal Arts 

0.66 

0.94 

0.28 

Liberal Arts 

0.62 

0.90 

0.28 

Liberal Arts 

0.58 

0.88 

0.30 

Liberal Arts 

0.49 

0.80 

0.31 

Liberal Arts 

0.88 

1.00 

0.12 

Liberal Arts 

0.76 

0.93 

0.17 

Liberal Arts 

0.94 

1.00 

0.06 

Liberal Arts 

0.92 

1.00 

0.08 

Liberal Arts 

0.82 

0.84 

0.02 

Liberal Arts 


Table 2: Descriptive Statistics 


College 

Mean Before 

S.D. Before 

Mean After 

S.D. After 

Mean Difference 

S.D. Difference 

Education 

0.839 

0.590 

0.933 

0.014 

0.094 

0.050 

Business 

0.757 

0.118 

0.915 

0.018 

0.158 

0.103 

L. Arts 

0.719 

0.167 

0.917 

0.022 

0.198 

0.122 

Combined 

0.772 

0.129 

0.922 

0.057 

0.150 

0.103 


The data were analyzed using a paired sample t-test to determine if the training received was effective. The 
null hypothesis is that the training program was not effective, while the alternative hypothesis is that it was effective. 
The test was conducted at the five percent level of significance. The results of this paired sample t-test are found in 
Table 3. 


Table 3: Results of Paired Sample T-Test 


Computed t 

df 

Critical t 

P-value 

7.99 

29 

+ 2.045 

0.000 


Since computed t is greater than critical t, the null hypothesis is rejected. The results from the analysis of 
the sample data suggest that the training increased the scores beyond that which could be attributed to random or 
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chance factors. It can be concluded that the Developmental teacher training Program significantly increased SAA 
scores. 


Another question of interest is if there is a difference in the mean change in SAA scores by college. The 
null hypothesis is that there is no difference in the mean change in SAA scores by college, while the alternative 
hypothesis is that there is a difference. The data were analyzed using a one-way analysis of variance (ANOVA). 
This test was also conducted at the five percent level of significance. The results of this one-way ANOVA are found 
in Table 4. 


Table 4: Results of One-Way ANOVA 


Computed F 

df 

Critical F 

P-value 

2.95 

2, 27 

3.354 

0.069 


The computed value of F is not greater than the critical value, so the null hypothesis cannot be rejected. The 

evidence from the sample suggests that there is no difference in the mean change in SAA scores by college. 

CONCLUSIONS 

1. Based on the sample and findings discussed herein, it appears that the Developmental teacher training 
program creates a statistically significant improvement in faculty ability to write course objectives 
compliant with Bloom's Taxonomy thus supporting the purpose of this research project. 

2. Based on before and after values, it appears that there is no statistically significant effect for the college in 
which the faculty member teaches. 

3. It should be noted that this analysis is based on a single session of the Developmental teacher training 
program containing a limited number of participants (n=30). Additional evaluations for subsequent 
program sessions are recommended to increase the number of participants and confirm the findings herein. 

REFERENCES 

1. Almerico, G.M. & Baker, R. (2005). Bloom’s Taxonomy illustrative verbs: Developing a comprehensive 
list for educator use. Journal of the Florida Association of Teacher Educators. 

2. Applying Bloom's Taxonomy. Retrieved April 14, 2004, from http://www.teachers.ash.org.au/ 
researchskills/dalton.htm 

3. Bloom, B. S. (1956). Taxonomy of Educational Objectives Handbook I: Cognitive Domain. New York: 
McKay. 

4. Bloom's Taxonomy. Retrieved April 14, 2004, from http://www.officeport.com/edu/blooms.htm 

5. Bloom’s Taxonomy. Retrieved April 14, 2004, from http://www.bus.ucf.edu/welch/fctl/new bloom.htm 

6. Bloom’s Taxonomy of Cognitive Levels. Retrieved April 14, 2004, from http://edtech.clas.pdx.edu/ 
presentations/frr99/blooms.htm 

7. Bloom's Taxonomy of Cognitive Levels. Retrieved April 14, 2004, from http://www.irn. pdx.edu/- 
perrinn/Eval/blooms.htm 

8. Bloom’s Taxonomy (Cognitive Domain), Words for Stating Performance Objectives. Retrieved April 14, 
2004, from http://www.udel.edu/educ/socstuds/cog hierarchy.htm 

9. Bloom's Taxonomy: Sample Questions. Retrieved April, 14, 2004, from http://www.officeport. 
com/edu/bloomci .htm 

10. Department of Education and Training, Government of Western Australia. Bloom’s Taxonomy. Retrieved 
April 14, 2004, from http://www.eddept.wa.edu. au/gifttal/EAGER/Bloom%27s %20Dara%20Wakefield. 
html#anchor45158 

11. Huitt, W. (2000). Bloom et. al’s Taxonomy of the Cognitive Domain. Retrieved April 14, 2004, from 
http://chiron.valdosta. edu/whuitt/col/cogsys/bloom.html 


30 





Contemporary Issues In Education Research - Fourth Quarter 2008 


Volume 1, Number 4 


12. 

Madison Area Technical College. (2003). Outlines of Instruction. Retrieved April 14, 2004, from 
httn://matcmadison.edu/is/iss/etla/curriculum/Outline of Instruction/verbs.htm 

13. 

Pearson Education, Inc. (2004). Bloom’s Taxonomy Verbs. Retrieved April 14, 2004, from 
httr>://www. teachervision.fen.com/lesson-Dlans/lesson-2172.html 

14. 

Sullivan, S. (1998). Bloom’s Taxonomy. Retrieved April 14, 2004, from 
httD://www.edschool.csuhavward.edu/DeDartments/ted/Sullivan/5099/Blooms.html 

15. 

Teach-nology, Inc. (2004). Bloom's Taxonomy Verbs. Retrieved April 14, 2004, from 
httD://www.teachnoloav. com/worksheets/time savers/bloom/ 

16. 

University of Memphis. (2003). Bloom’s Taxonomy-Action Verbs Requiring Cognitive Outcomes (Ideas) 
Retrieved April 14, 2004, from 

httD://www.DeoDle.memDhis.edu/~e 2 holson/ 11 nitus/clar/BLOOMSTAXl.HTM 

17. 

University of Washington (2001). Major Categories in the Taxonomy of Educational Objectives (Bloom 
1956). Retrieved Aoril 14, 2004, from httD://facultv.washin 2 ton.edu/~krumme/ 2 uides/bloom.html 


APPENDIX 
Workshop Handout 

This is a workshop designed to help instructors develop better course objectives. The workshop focuses on 
using Bloom's Taxonomy to develop specific course objectives in the ABCD (audience, behavior, condition, degree) 
format. Attendees are invited to bring their course objectives and syllabi for review. 

Workshop Task: 

1. Listed below in Table 1 are Bloom’s Taxonomy action verbs used to develop learning objectives and define 
student assignments. For each of the six taxonomy categories, select five action verbs and record them on 
the worksheet provided. 

2. Table 2 provides a list of possible assignment products from various fields of study. Select an assignment 
product from Table 2 that corresponds to each of the action verbs you recorded for each of the six Bloom’s 
Taxonomy categories. 
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Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluation 

Arrange 

Add 

Acquire 

Advertise 

Abstract 

Appraise 

Cite 

Approximate 

Adapt 

Analyze 

Animate 

Argue 

Define 

Articulate 

Allocate 

Audit 

Anticipate 

Assess 

Delineate 

Associate 

Alphabetize 

Blueprint 

Assemble 

Attach 

Describe 

Clarify 

Apply 

Breadboard 

Budget 

Compare 

Draw 

Classify 

Ascertain 

Break Down 

Code 

Conclude 

Duplicate 

Convert 

Assign 

Categorize 

Collect 

Core 

Enumerate 

Detail 

Attain 

Characterize 

Combine 

Counsel 

Find 

Elaborate 

Avoid 

Confirm 

Compile 

Criticize 

Identify 

Estimate 

Back Up 

Contrast 

Compose 

Critique 

Index 

Exemplify 

Build 

Correlate 

Construct 

Decide 

Indicate 

Express 

Calculate 

Debate 

Cope 

Defend 

Isolate 

Extend 

Capture 

Deduce 

Correspond 

Dispute 

Know 

Extrapolate 

Change 

Detect 

Craft 

Editorialize 

Label 

Factor 

Choose 

Diagnose 

Create 

Evaluate 

List 

Give 

Chose 

Diagram 

Cultivate 

Grade 

Locate 

Infer 

Complete 

Differentiate 

Debug 

Hire 

Match 

Interact 

Compute 

Discover 

Depict 

Judge 

Meet 

Interpret 

Demonstrate 

Discriminate 

Design 

Justify 

Memorize 

Observe 

Depict 

Dissect 

Develop 

Measure 

Met 

Paraphrase 

Depreciate 

Distinguish 

Devise 

Prescribe 

Name 

Picture 

Derive 

Ensure 

Dictate 

Prioritize 

Point to 

Predict 

Determine 

Examine 

Discuss 

Rank 

Quote 

Put In Own Words 

Diminish 

Experiment 

Engineer 

Rate 

Read 

Reorder 

Dramatize 

Explain 

Enhance 

Recommend 

Recall 

Rephrase 

Employ 

Explore 

Explain 

Release 

Recite 

Report 

Examine 

Figure Out 

Facilitate 

Review 

Recognize 

Restate 

Exercise 

File 

Forecast 

Revise 

Record 

Retell 

Exhibit 

Group 

Format 

Score 

Recount 

Subtract 

Expose 

Inquire 

Formulate 

Support 

Relate 

Summarize 

Factor 

Inspect 

Generalize 

Test 

Repeat 

Trace 

Figure 

Interrupt 

Generate 

Validate 

Reproduce 

Vary 

Graph 

Inventory 

Handle 

Value 

Select 

Visualize 

Handle 

Investigate 

Hypothesize 

Verify 

Show 


Illustrate 

Lay Out 

Imagine 

Weigh 

Study 


Interpolate 

Maximize 

Improve 


Tabulate 


Interview 

Minimize 

Incorporate 


Tell 


Made 

Optimize 

Individualize 


Underline 


Make 

Order 

Initiate 




Manipulate 

Organize 

Integrate 




Modify 

Outline 

Interface 




Operate 

Point Out 

Invent 




Plot 

Query 

Lecture 




Portray 

Question 

Model 




Practice 

Relate 

Modify 




Price 

Separate 

Originate 




Process 

Size Up 

Overhaul 




Produce 

Solve 

Plan 




Project 

Specify 

Portray 




Protect 

Subdivide 

Pose 




Provide 

Survey 

Prescribe 




Put Into Practice 

Test 

Program 




Round Off 

Train 

Propose 




Schedule 

Transform 

Rearrange 
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Table 1: Assignment Verbs 
continued 

Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluation 



Sequence 

Utilize 

Reconstruct 




Show 

Transfer 

Refer 




Simulate 


Reinforce 




Sketch 


Reorganize 




Solve 


Report 




Subscribe 


Revise 




Tabulate 


Rewrite 




Teach 


Schematize 




Transcribe 


Set Up 




Translate 


Speculate 




Use 


Support 






Systematize 




Table 2: Assignment products 


Advertisement 

Illustrations 

Poster Board 

Analogies 

Income statement 

Program 

Balance sheet 

Internal rate of return 

Questionnaires 

Book reviews 

Interviews 

Radio shows 

Cartoon 

Inventions 

Recipes 

Cash flow statement 

Journal 

Recommendations 

Chart 

Laws 

Recordings 

Circuit 

Lessons (student taught) 

Report 

Collages 

Letter 

Research 

Comic strips 

Machines 

Resume 

Commercials 

Map 

Return on investment 

Confessions 

Market analysis 

Role playing 

Construction 

Metaphor 

Scrap book 

Conclusions 

Model 

Screen play 

Critiques 

Movie review 

Sculpture 

Dances 

Murals 

Self evaluation 

Database 

Myths 

Short story 

Debates 

Net present value 

Sketches 

Design plans 

Network 

Songs 

Diagrams 

News article 

Spreadsheet 

Diaries 

Outlines 

Speculations 

Dictionaries 

Painting 

Speeches 

Dioramas 

Panel discussion 

Strategic analysis 

Displays 

Pantomime 

Survey 

Essays 

Petition 

Term paper 

Forecast 

Photograph 

Theater performance 

Formula 

Picture 

Time line 

Games 

Poem 

Valuing 

Graphs 

Portfolio 

Worksheet 

Taxonomy Level 

Action Verbs 

Assignment Products 

Knowledge 



Comprehension 



Application 



Analysis 



Synthesis 



Evaluation 
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3. For each pair of corresponding verbs and products, write a course objective in Bloom’s Taxonomy 
audience behavioral, condition, degree/criteria (ABCD) format. 

Example: Using a battery, wire, switch and bulb. Electrical Circuits 1 students will create an electrical circuit 

such that the bulb will light when the switch is turned on. 

Audience - Electrical Circuits 1 students 
Behavior - will create 

Condition - using a battery, wire, switch and bulb 
Degree/criteria - bulb will light when the switch is turned on 


Course objectives by Taxonomy level 

Knowledge 

Comprehension 

Application 

Analysis 

Synthesis 

Evaluation 


NOTES 
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